Human visibility is limited and it is impossible for a single person to visually capture the entire directions in a short period of time. If a person is required to visually capture a range beyond the human visibility (hereinafter referred to as a wide range), a special monitoring system is required. As a technique used for this special system, there is known a technique to pick up (photograph) an image as an object to be monitored using an image pickup device for wide range picking-up (referred to as a wide range image pickup device) so as to enable a wide range observation by an image (hereinafter referred to as a wide range image) in which the picked-up image was processed to fit in the visibility.
As a wide range image pickup device, there is an image pickup device capable of picking up an image in a wide range with a single device such as a panorama camera. There is another wide range image pickup device in which images picked up by a plurality of image pickup devices are combined to provide a wide range image. An observer sees the wide range image displayed in a display device so that a range beyond the visibility can be visually captured.
In the case of displaying a wide range image with a sufficient image size and resolution, a screen size required for displaying the image is increased in a display device. Therefore, a large-sized display device capable of displaying the wide range image is required to monitor an object to be monitored using the wide range image.
If it is difficult to provide a large-sized display device, there is a technique to display the wide range image in a small-sized display device by decreasing the image size (reducing resolution). A technique is known to achieve the wide range observation even in a smaller sized system by using a reduced image for monitoring. This technique enables the reduction of costs used for constructing a system. In addition, construction of a system to monitor a wide range in a limited space can be achieved.
If a wide area image is displayed after being reduced, there is a case that information required for monitoring is difficult to obtain from the reduced image (hereinafter referred to as a reduced display image). In this case, there is known a technique in which a desired amount of information can be obtained by increasing the resolution and enlarging the display in a specified region (hereinafter referred to as a specific region) of the reduced display image.
The technique described above is disclosed, for example, as a visual information providing device in Japanese Laid Open Patent Application JP-A-Heisei 5-139209. This visual information supplying device includes a first generating means, a first display means, a second generating means, a second display means, a presenting means, a switch signal generating means, and an image control device. The first generating means generates first visual information. The first display means displays the first visual information as a first image. The second generating means generates second visual information composed of information for the right eye and information for the left eye. The second display means displays the second visual information as a three-dimensional second image. The presenting means is capable of presenting the first and second images independently or by superposing. The switch signal generating means outputs a switch signal to select an image presented in the presenting means. The image control device controls the presenting means on the basis of the switch signal.
In order to realize an enlarged display, the specific region described above needs to be selected. Selection of the specific region has been generally made by operating a changeover switch externally provided for the display device. In a system which does not have the external changeover switch, a specific point on the screen seen by an observer (hereinafter referred to as a visual point) is detected so that a region corresponding to the visual point is made to be a specific region. In the JP-A-Heisei 5-139209, two of a high resolution display device and a low resolution display device are superposed for being used as described above. The sight line direction of the observer is detected so that an image in a region of the sight line direction is displayed in the high resolution display device while displaying a region in the vicinity thereof in the low resolution display device.
In the above described system to detect the sight line direction, an eyeball direction (sight line) detecting sensor for detecting the sight line of the observer is required. The sight line detecting sensor is, for example, mounted on the head of the observer, in which the inclination of the head is detected in response to the movement so as to specify the visual point from the inclination. In another example, the visual point is specified in response to the movement of the eyeballs by mounting the sight line detecting sensor to the observer in such a manner to wear glasses. A technique using the latter method is described in, for example, Japanese Laid Open Patent Application JP-A-Heisei 9-305156.
The JP-A-Heisei 9-305156 discloses a video display method and device. This video display method is a video display method to create and display accurate video at a high speed. It is characterized in that video is created with a normal accuracy in whole video to be displayed, the sight line is continuously detected by using an eyeball direction detecting means, a position on the video corresponding to the center of the visibility is calculated in each moment, video near the center of the visibility is created on the basis of the calculation result by using at least one selected from a high resolution with increased display pixels and a high reproduction with increased colors and gradations.
In the technique described in the JP-A-Heisei 5-139209, a high resolution image is created exclusively for a region in the sight line direction obtained from the eyeball direction detection in order to accelerate the image display. In the techniques described in the JP-A-Heisei 5-139209 and the JP-A-Heisei 9-305156, the sight line direction of the observer is automatically detected by the eyeball movement estimating means, so that a region in the sight line direction is displayed in the high resolution. Therefore, a part required by the observer is automatically displayed in the high resolution without conducting a switch operation.
There is a demand for a technique to appropriately present a wide range image exceeding the visibility of the observer and multiple visual point images within the visibility of the observer. A technique is also demanded to appropriately present a wide range image exceeding the visibility of the observer and multiple visual point images within the visibility of the observer without a special device for detecting the movement of the eyeballs to be mounted to the observer.
As related techniques, an image pickup device is disclosed in Japanese Laid Open Patent Application 2000-59665A. This image pickup device is composed of three image pickup units in which two of the image pickup units are provided on both left and right sides of the one image pickup unit. In the image pickup device, the optical axes have a relationship, in which each of optical axes of the image pickup unit on the left side and the image pickup unit on the right side intersects with an optical axis of the center image pickup unit in the same angle, and these intersecting points of the optical axes are the same point. The three image pickup units are positioned in the same interval in the distance from the each of the intersecting points of the optical axes to the front principal point of each of the image pickup units, in the directions close to an object side from the intersecting points of the optical axes.
An image processing method and device is disclosed in Japanese Laid Open Patent Application JP-A-Heisei 8-116556. This image processing method is characterized in that it includes a multiple visual point image input step for inputting images obtained from a plurality of visual point positions arranged on a plurality of different straight lines, a visual point detecting step for detecting a position of eyes of an observer seeing the images and a direction to be seen, an image reconstruction step for reconstructing an image to be seen from the visual point position detected in the view point detecting step from multiple visual point images data, and an image output step for outputting the reconstructed image via an image output device.